20949P-000800US 

George B- F Yee, Reg. No. 37,478 

Telephone: 650-326-2400 

Inventors: Richard Meyer et al. 

Title: Failover Processing in a Storage System 

Sheets of drawings 1 of 36 



| SNMP | 1 Alarm | | Logging | 



4^ 



Error Analysis 



Error Recovery Sequence 



Error Statistics 



Reliability Rating 



| Error Recovery 
Resource i-M Table 



Request Error 
Analysis 



Take Corrective 
Action 



Report Corrective 
^_Action Results 



Reconstruct 
RAID 
Member 



Run 
Diagnostic 



Shutdown | 



Define Error Types 
and Recovery 
Actions for Resource 



Status 
Reporting j 



Resource 
State Change 
Commands 



Report 
Status X-Z 



Resource Manager 



| j~Over-T< 



\ IP Path Errors j 
| Hardware Error | 



Processor 
Exception or 
Trap 



Error 
Collection 



Figure 1 - Error Recovery Architecture 
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Figure 2 - Non-Fault Tolerant Configuration 
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Figure 3 - Fault Tolerant Configuration 
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Figure 4 - High Availability Configuration 
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Figure 6: Member Unit State Diagram 
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Figure 7 - Creating a Failover Set 
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Figure 8 - Member State Diagram 
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Figure 9 - Member Arbitration for COLD Boot 
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Figure 10 - Member Arbitration for WARM Boot 
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Figure 11 - Member Arbitration for Mixed Boot 
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Figure 14: Served Failover Set State Machine Diagram 
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Figure 17 - Transmitter fault (sends a bad value) 
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Figure 18 - Transmitter fault (doesn't send a value) 
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Figure 19 - Receiver fault (relays wrong value) 
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Figure 20 - Receiver fault (doesn't relay a value) 
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Figure 21 A - Failover Service Architecture 
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Figure 21B 
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Figure 22 - An Arbiter for the Database 
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Figure 24 - VSX to VSX Message Passing 



20949P-000800US 

George B. F. Yee, Reg. No. 37,478 

Telephone: 650-326-2400 

Inventors: Richard Meyer et al. 

Title: Failover Processing in a Storage System 

Sheets of drawings 23 of 36 




Server Server 




Disk array Disk Disk array 



Figure 26 - Shared Disk 



20949P-000800US 

George B. F. Yee, Reg. No. 37,478 

Telephone: 650-326-2400 

Inventors: Richard Meyer et al. 

Title: Failover Processing in a Storage System 

Sheets of drawings 24 of 36 



Source 
(1) 


Message Passing 
Interface (1) 


SCSI 
Manager(1) 


SCSI 
Manager (2) 


Message Passing 
Interface (2) 


Destination 
(2) 


Send MSC 
to\ 


from VSX (1) 
'SX (2) 

I/O 
I/O 


Request 

Writ 
E 

Res pons 

-4 

Done 

Re 
E 

Respon 


i Buffer 
id 

e(status) 

I/O Rc 

id Buffer 
nd 

;e(status) 

l/OC 


Get Me 

quest 
one 

Deliver tv 


ssage [p°"s] 
essage 



Figure 27 - VSX to VSX communication Using Shared Disk 
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Figure 28 -2 Node HA Configuration 
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Figure 29 - Hierarchical HA Configuratic 
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Figure 31 - N - Nodes 
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Figure 32 - VSX Failover, Primary Fails 
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Figure 33 - IO Path Failover - LC Fails 
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1. FCASIC Crashes 

2. LC detects FC ASIC crash 

3. Error Analysis determines IO Path fails for all devices (server or storage) on FC ASIC 

4. Upstream hLUNs report CHECK CONDITION for all devices connected to failed FC Ports. RCON 
and FORMAT aborted, if necessary 

5. Restart RCON and FORMAT, if necessary 



Figure 34 - IO Path Failover - FC Port Fails 
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1 . Link down on port 

2. LC detects FC Port link down 

3. Error Analysis determines IO Path fails for all devices (server or storage) on FC Port 

4. Upstream hLUNs report CHECK CONDITION for all devices connected to FC Port. RCON 
and FORMAT aborted, if necessary 

5. Restart RCON and FORMAT, if necessary 



Figure 35 - IO Path Failover - Link Down 
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Figure 36 A Physical Setup for VSX-HA - Variation 1 
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Figure 36 C Physical Setup for VSX-HA - Variation 3 
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Figure 36 D Physical Setup for VSX-HA - Variation 4 
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